# Chinese Pre-training

ERNIE-4.5-0.3B-PT-bf16 is a version of the ERNIE series models developed by Baidu, with a parameter scale of 0.3B and trained using bf16 precision.

MiniRBT is a Chinese small pre-trained model developed based on knowledge distillation technology, optimized for training efficiency using Whole Word Masking.

MiniRBT is a small Chinese pre-trained model based on knowledge distillation technology, combined with whole word masking, suitable for various Chinese natural language processing tasks.

StructBERT is a novel model that extends BERT by incorporating linguistic structures into the pre-training process, leveraging two auxiliary tasks to fully utilize word and sentence order structures

A Chinese tokenized version of the RoBERTa medium model pre-trained on CLUECorpusSmall corpus, with tokenization processing to enhance sequence handling efficiency

PERT is a Chinese pre-trained model based on BERT, focusing on improving Chinese text processing capabilities.

Chinese ELECTRA is a pre-trained model developed by the Harbin Institute of Technology-iFLYTEK Joint Lab (HFL) based on the ELECTRA model released by Google and Stanford University. It features a small parameter size and high performance.

Chinese ELECTRA is a pre-trained model developed by the Harbin Institute of Technology-iFLYTEK Joint Lab based on Google's ELECTRA architecture, with only 1/10 the parameters of BERT but comparable performance.

ChineseBERT is a Chinese pre-training model that integrates glyph and pinyin information, enhancing Chinese comprehension through improved glyph features

A Chinese word-based RoBERTa medium model pre-trained on CLUECorpusSmall, featuring an 8-layer 512-hidden architecture with superior performance and faster processing speed compared to character-based models